MLOps: Deploying Models into Production

Myles Mitchell @ Jumping Rivers

Welcome!

Check in using the QR code!

QR code for afternoon session check-in

Virtual environment

https://mlops-intro.jumpingrivers.training/welcome/

Password: pomelo-jujube

Screenshot of log in page

Before we start…

Who am I?

  • Background in Astrophysics.

  • Data Scientist @ Jumping Rivers:

    • Python & R support for various clients.

    • Teach courses in Python, R, SQL, Machine Learning.

  • Hobbies include hiking and travelling.

Jumping Rivers

↗ jumpingrivers.com   𝕏 @jumping_uk

  • Machine learning
  • Dashboard development
  • R packages and APIs
  • Data pipelines
  • Code review
     

How will this workshop run?

  • Presentation slides (bit.ly/2024-nicd-mlops).

  • R coding demos in virtual environment.

  • Exercises will apply the demo code to a different dataset and model.

  • Not an R user?

    • The complete scripts and solutions are available.
    • Not all exercises will be code-based.

What will be covered?

  • Introduction to MLOps

  • Building a basic MLOps workflow

  • We will be deploying models locally only…

    • Means all tools are opensource and free!

  • Want to contribute?

    • Raise your hand for questions and comments (bonus points…).

What you get from me

Discussion

  • What is MLOps?

  • Discuss

  • Extra points if you contribute…

  • QR Code:

03:00

Introduction to MLOps

Let’s take a step back…

The typical data science workflow:

Typical data science workflow. Starting with data importing and tidying, followed by a cycle of data transformation, data visualisation and modelling which repeats as the model is better understood. The results from this cycle are then communicated.
  • Data is imported and tidied.
  • Cycle of data transformation, data visualisation and modelling.
  • The cycle repeats as we understand the underlying model better.
  • The results are communicated to an external audience.

From Classical Stats to Machine Learning

  • The classical workflow prioritises understanding the system behind the data.
  • By contrast, Machine Learning prioritises prediction.
  • As data grows we reconsider and update our ML models to optimise predictive power.
  • A goal of MLOps is to streamline this cycle.

What is MLOps?

MLOps: Machine Learning Operations

MLOps workflow. Starting with data importing and tidying, followed by modelling and finishing with model versioning, deployment and monitoring. The cycle then repeats as more data is acquired.
  • Framework to continuously build, deploy and maintain ML models.
  • Encapsulates the “full stack” from data acquisition to model deployment.
  • Monitor models in production and detect “model drift”.
  • Versioning of models and data.

MLOps frameworks

  • Amazon SageMaker
  • Google Cloud Platform
  • Kubeflow (ML toolkit for Kubernetes)
  • Vetiver by Posit (free to install, nice for beginners)
  • And the list goes on…

     

Vetiver

  • Opensource tool maintained by Posit (formerly RStudio).
  • Integrates with popular ML libraries in R and Python.
  • Fluent tooling to version, deploy and monitor a trained model.
  • Supports deploying models to localhost - great way to learn MLOps!

Your first MLOps pipeline

Let’s build an MLOps stack!

  • Data
  • Modelling
  • Deployment
  • Monitoring
  • Repeat

Modelling penguins!

Let’s set up a basic MLOps workflow!

  • Palmer Penguins dataset:

    library("palmerpenguins")

names(penguins)
    [1] "species"           "island"            "bill_length_mm"   
[4] "bill_depth_mm"     "flipper_length_mm" "body_mass_g"      
[7] "sex"               "year"
Scatter plot showing positive relationship between penguin flipper length and penguin body mass. The data points are coloured based on species and shaped based on island. The Gentoo penguins tend to have higher body mass and flipper length than Adelie and Chinstrap.

Palmer Penguin dataset

Data tidying and cleaning

Let’s predict species using flipper length, body mass and island!

  • Using {tidyr} and {rsample}:

    # Drop missing data
penguins_data = tidyr::drop_na(penguins)

# Split into train and test sets
penguins_split = rsample::initial_split(
  penguins_data, prop = 0.8
)
train_data = rsample::training(penguins_split)
test_data = rsample::testing(penguins_split)

Modelling penguins!

Open demo.R in RStudio…

Task 1: Data loading and tidying

  • Open exercises.R

  • Attempt “Task 1: Data loading and tidying”

  • Need help? Check demo.R or raise your hand

  • Not an R user? The solution can be found in solutions.R

  • Finished? Scan the QR code for extra points!

05:00

Data best practices

Loading and importing

  • Consider moving from large CSV files to a more efficient format like Apache Parquet and Apache Arrow.
  • Add a data validation check to prevent unexpected data formats entering the pipeline.
  • Consider tools like Apache Spark for fast processing of Big Data.

Data best practices

Tidying & cleaning

  • Consider creating an R package to encourage proper documentation, testing and dependency management.
  • Optimise bottlenecks to improve efficiency.
  • Split into training and validation sets.

Data best practices

Versioning

  • Ensuring reproducibility is vital
    • Example: why did your model have poor accuracy for ethic minority background individuals?
  • Include timestamps in your database queries.
  • Ensure your training set can be retrieved in the future.

Data best practices

Take advantage of native tools on ML platforms

  • Your preferred ML platform probably has built-in tools for data wrangling!
    • SageMaker Data Wrangler
    • H2O.ai feature engineering (no code)

Modelling penguins!

  • Let’s set up the model recipe in {tidymodels}:
library("tidymodels")

model = recipe(
  species ~ island + flipper_length_mm + body_mass_g,
  data = train_data
) |>
  workflow(nearest_neighbor(mode = "classification")) |>
  fit(train_data)

Modelling penguins!

Our model object can now be used to predict species:

model_pred = predict(model, test_data)

# Accuracy for unseen test data
mean(
  model_pred$.pred_class == as.character(
    test_data$species
  )
)
[1] 0.8059701

Enter Vetiver!

  • Use a Vetiver model object to collate all of the info needed to store, deploy and version our model:

    v_model = vetiver::vetiver_model(
  model,
  model_name = "k-nn",
  description = "penguin-species"
)
v_model
    
── k-nn ─ <bundled_workflow> model for deployment 
penguin-species using 3 features

Vetiver model

v_model is a list with six elements

  • View the contents:

    names(v_model)
    [1] "model"       "model_name"  "description" "metadata"    "prototype"  
[6] "versioned"

  • View the model description:

    v_model$description
    [1] "penguin-species"

  • View the metadata:

    v_model$metadata
    $user
list()

$version
NULL

$url
NULL

$required_pkgs
[1] "kknn"      "parsnip"   "recipes"   "workflows"

Demo

Task 2: Modelling

  • Open exercises.R (and rerun your Task 1 code)

  • Attempt “Task 2: Modelling”

  • Need help? Check demo.R or raise your hand

  • Not an R user? The solution can be found in solutions.R

  • Finished? Scan the QR code for extra points!

05:00

Modelling best practices

Choosing the right model can be tough!

Modelling best practices

Versioning

  • Store scoring metrics and model parameters from each experiment.
  • Any previously-deployed model should be retrievable, along with the data used to train it.

Model versioning

  • Use {pins} to store R or Python objects for reuse later.

  • Store pins using “boards” including Posit Connect, Amazon S3 or even Google drive!

  • Storing in a temporary directory:

    model_board = pins::board_temp(versioned = TRUE)
model_board |> vetiver::vetiver_pin_write(v_model)

Retrieving a pinned model

  • Retrieve a model

    model_board |> vetiver::vetiver_pin_read("k-nn")
    
── k-nn ─ <bundled_workflow> model for deployment 
penguin-species using 3 features

  • Inspect the stored versions

    model_board |> pins::pin_versions("k-nn")
    # A tibble: 1 × 3
  version                created             hash 
  <chr>                  <dttm>              <chr>
1 20240927T083148Z-fe6b9 2024-09-27 09:31:48 fe6b9

Model deployment

Deployment using Vetiver

  • We deploy models as APIs which take input data and send back model predictions.

  • We can use a {plumber} API to deploy a {vetiver} model.

Deploying locally

  • {vetiver} and {plumber} support local deployment:

    plumber::pr() |>
  vetiver::vetiver_api(v_model) |>
  plumber::pr_run()

  • Opens the API in a browser window

  • Great for beginners to MLOps and APIs!

Deploying locally

Check the deployment with:

base_url = "127.0.0.1:8080/"  # update the port number!
url = paste0(base_url, "ping")
r = httr::GET(url)
metadata = httr::content(
  r, as = "text", encoding = "UTF-8"
)
jsonlite::fromJSON(metadata)

Model predictions

Checking that our API works!

  • Endpoints metadata and predict allow programmatic queries:

    url = paste0(base_url, "predict")
endpoint = vetiver::vetiver_endpoint(url)
pred_data = test_data |>
  dplyr::select(
    "island", "flipper_length_mm", "body_mass_g"
  ) |>
  dplyr::slice_sample(n = 10)
predict(endpoint, pred_data)

Demo

Task 3: Deploying your model

  • Open exercises.R

  • Attempt “Task 3: Deploying your model”

  • Need help? Check demo.R or raise your hand

  • Not an R user? The solution can be found in solutions.R

  • Finished? Scan the QR code for extra points!

05:00

Aside: What about Python?

  • Vetiver is available for both Python and R!

  • In Python you would use Python ML libraries rather than {tidymodels}

    • scikit learn
    • PyTorch
    • XGBoost
    • statsmodels

  • Vetiver documentation: vetiver.posit.co

Deployment best practices

  • Try deploying locally to check that your model API works as expected.

  • Use environment managers like {renv} to store model dependencies.

  • Use containers like Docker to bundle model source code with dependencies.

Deploying to the cloud

  • Vetiver also streamlines deployment to the production environment:

    vetiver::vetiver_prepare_docker(
  pins::board_connect(), 
  "myles/k-nn", 
  docker_args = list(port = 8080)
)

  • This command:

    • Lists R depedencies with {renv}

    • Stores the {plumber} API code in plumber.R

    • Generates a Dockerfile

Docker files

Our Dockerfile contains a series of commands to:

  • Set the R version and install the system libraries.

  • Install the required R packages.

  • Run the API in the deployed environment.

Running Docker

  • Build a Docker container:

    docker build --tag my-first-model .

  • Inspect your stored Docker images:

    docker image list

  • Run the image:

    docker run --rm --publish 8080:8080 my-first-model

  • These steps can be run in sequence using a CI/CD pipeline.

Deploying to Connect

  • Vetiver integrates nicely with Posit Connect:

    vetiver::vetiver_deploy_rsconnect(
  board = pins::board_connect(), "myles/k-nn"
)

  • We can also publish to Amazon SageMaker using vetiver_deploy_sagemaker()

Cost considerations for cloud MLOps

Cloud MLOps normally doesn’t come free…

  • Some platforms offer free trials (e.g., SageMaker).

  • May be cheaper if you’re already invested in a particular cloud platform

    • Data services
    • App deployment

  • Costs can rise depending on computational resources consumed.

  • Model building and deployment use different environments.

Monitoring your model

Deployment is just the beginning…

Discussion - Model monitoring

  • Why should a deployed model be closely monitored?

  • What warning signs would you look out for?

  • Discuss

  • Bonus points if you contribute…

  • QR Code:

03:00

Model drift

  • Model performance may drift as the data evolves…
    • Data drift: statistical distribution of input feature changes.
    • Concept drift: relationship between target and input variables changes.
  • The context in which your model was trained matters!

Task 4: Detecting model drift

  • Open exercises.R

  • Attempt “Task 4: Detecting model drift”

  • Need help? Check demo.R or raise your hand

  • Not an R user? The solution can be found in solutions.R

  • Finished? Scan the QR code for extra points!

05:00

Model monitoring

  • As your data grows, run regular checks of model performance.

  • Monitor key model metrics over time.

  • You may notice a downward trend…

  • Retrain the model with the latest data and redeploy.

Monitoring demand

As data and user base grows, your model needs to scale.

  • Upgrade your computational resources.

  • Consider moving from a relational database to a data warehouse.

  • Check how many users your license (AWS, Posit, etc) permits.

Monitoring with Vetiver

Vetiver has built in functions to track scoring metrics over time.

  • Requires a time variable in the dataset.

  • Load the model from your {pins} board.

  • Make sure you are scoring the deployed version.

  • Specify the period for scoring (weeks, months, years, …).

  • Model metrics can also be stored with {pins}!

Monitoring with Vetiver

Consider our life expectancy data from the exercises…

  • Compute scoring metrics over specified period:

    new_metrics = vetiver::augment(
  v_model, new_data = recent_data
) |>
  vetiver::vetiver_compute_metrics(
    Date, "year", `Life expectancy`, .pred
  )

  • Requires a Date column (generate from Year).

  • recent_data: could be data from the past year or all historical data.

Monitoring with Vetiver

  • Pin the metrics

    model_board |>
  vetiver::vetiver_pin_metrics(
    new_metrics, "k-nn_metrics", overwrite = TRUE
  )

  • Plot the metrics

    library("ggplot2")

monitoring_metrics = model_board |>
  pins::pin_read("k-nn_metrics")
vetiver::vetiver_plot_metrics(monitoring_metrics) +
  scale_size(range = c(2, 4))

Closing thoughts

Pros / cons of MLOps

  • Retraining and redeployment can happen at the click of a button.

  • Encourages good practices like model versioning and packaging of source code.

  • Reduces human error.

  • Well defined and reproducible.

  • Consider whether it is worth the cost/effort before starting.

Thanks for listening!

  • Remember to provide feedback!

  • Open this workshop in the conference app

  • Tap “More”

  • Tap “Feedback”

  • Fill in the form

Workshop QR code

QR code for afternoon session check-in